Multi-objective Analysis of MAP-Elites Performance

In certain complex optimization tasks, it becomes necessary to use multiple measures to characterize the performance of di erent algorithms. This paper presents a method that combines ordinal e ect sizes with Pareto dominance to analyze such cases. Since the method is ordinal, it can also generalize across di erent optimization tasks even when the performance measurements are di erently scaled. Through a case study, we show that this method can discover and quantify relations that would be di cult to deduce using a conventional measure-by-measure analysis. This case study applies the method to the evolution of robot controller repertoires using the MAP-Elites algorithm. Here, we analyze the search performance across a large set of parametrizations; varying mutation size and operator type, as well as map resolution, across four different robot morphologies. We show that the average magnitude of mutations has a bigger e ect on outcomes than their precise distributions. CCS CONCEPTS •Computing methodologies→ Search methodologies; Evolutionary robotics; •Mathematics of computing→ Nonparametric statistics; •General and reference→ Performance;